Deep Learning: Autodiff, Parameter Tying and Backprop Through Time∗
نویسنده
چکیده
How to do parameter tying and how this relates to Backprop through time.
منابع مشابه
Learning representations through stochastic gradient descent in cross-validation error
Representations are fundamental to artificial intelligence. The performance of a learning system depends on the type of representation used for representing the data. Typically, these representations are hand-engineered using domain knowledge. More recently, the trend is to learn these representations through stochastic gradient descent in multi-layer neural networks, which is called backprop. ...
متن کاملLearning Many Related Tasks at the Same Time with Backpropagation
Hinton [6] proposed that generalization in artificial neural nets should improve if nets learn to represent the domain's underlying regularities . Abu-Mustafa's hints work [1] shows that the outputs of a backprop net can be used as inputs through which domainspecific information can be given to the net . We extend these ideas by showing that a backprop net learning many related tasks at the sam...
متن کاملCrossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural Networks
Representations are fundamental to artificial intelligence. The performance of a learning system depends on the type of representation used for representing the data. Typically, these representations are hand-engineered using domain knowledge. More recently, the trend is to learn these representations through stochastic gradient descent in multi-layer neural networks, which is called backprop. ...
متن کامل2 MECHANISMS OF MULTITASK BACKPROPWe
Hinton 6] proposed that generalization in artiicial neural nets should improve if nets learn to represent the domain's underlying regularities. Abu-Mustafa's hints work 1] shows that the outputs of a backprop net can be used as inputs through which domain-speciic information can be given to the net. We extend these ideas by showing that a backprop net learning many related tasks at the same tim...
متن کاملDeep Generative Stochastic Networks Trainable by Backprop
We introduce a novel training principle for probabilistic models that is an alternative to maximum likelihood. The proposed Generative Stochastic Networks (GSN) framework is based on learning the transition operator of a Markov chain whose stationary distribution estimates the data distribution. The transition distribution of the Markov chain is conditional on the previous state, generally invo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015